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FOREWORD 


A perennial  problem  vexing  the  experimenter  is  that  of 
rejection  of  suspected  dataD  *or  on©  hundred  years  attempts 
at  the  solution  of  this  problem  have  been  advanced B most  ©f 
them  t©  be  themselves  rejected  as  suspeeto  Fortunately  xnod«= 
ern  statistical  theory  has  proposed  useful  reliable  methods 
for  objectively  rejecting  deviant  values0  However  the  solution 
Is  far  from  complete  at  prasento 

This  report  describee  for  the  experimenter  two  of  the 
modern  statistical  tests  available  for  possible  rejection 
of  outlying  observations 0 Th@s©  two  methods  hav©  been 
selected  because  they  apply  in  a majority  of  the  actually 
occurring  situations^  and  because  they  are  so  easy  to  &pplyG 
Tii©  report  liras  originally  motivated  by  Mr«  Prosehan9®  consult® 
ativ©  work  in  the  Ordnance  Development  Division^  national 
Bureau  ©f  Standards*,  However  it  is  applicable  more  generally 
to  the  scientific  and  engineering  work  at  th©  National  Bureau 
of  Standard a o 

It  should  bo  borne  in  mind  that  this  report^  although 
Intended  .for  practical  every-day  use  in  our  laboratories*,  Is 
not  the  final  word  ©n  th©  subject  of  rejection  of  suspected 
observations o 

Jo  Ho  Curtiss 

Chiefs  National  Applied 

Mathematics  Laboratories 

Ao  Vo  As tin  : 

Acting  Director 

National  Bureau  of  Standards 
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HOW  TO  DECIDE  OBJECTIVELY  WHETHER 
AN  OUTLYING  OBSERVAW5JT  SHOULD  BE  REJECTED 

Frank  Proschan 

Id  PROBLEM;  Here  is  a very  common  problem  facing  ex® 
perlmentbra  at  the  National  Bureau  of  Standards 0 The  typical 
scientist*  call  him  Perry  Men  ter  0 makes  a number  !^&ay  five)  • 
of  repeated  measurements  of -some  unknown  quantity©  On©  of 
the  values  (say  the  largest)  is  s©  far  removed  from  the  other 
four  that  he  suspects  that  it  may  b©  in  error o However*  Perry 
has  no  specific  knowledge  that  a mistake  actually  did  occur 0 
Let  us  assume * too*  that  h©  has  no  previous  data  from  which 
to  estimate  the  precision  of  measurement©  How  can  h©  decide* 
from , the  v alue a the ms  el  v©  3 » whether  the  suspected  value  is 
in  error  or  not? 

The  answer  seems  clears  He  should  consider  the  suspected 
value  as  in  error  when  it  seems  t@o  far  from  th©  other  four 
valitoBo  But  how  can  h©  judge  when  it  is  51  too  far  from  th© 
other  four  values75? 

1®2  LOGICAL  APPROACH s Hera  is  a simple*  logical*  objective 
criterion © Suppose  Perry  could  somehow  make  millions  of  sets 
of  five  observations  each©  Suppose*  too*  that  he  could  guarantee 
that  none  of  th©s®  observations  had  any  mistakes©  Call  a 
typical  set  Xy  Xy  Xy  Xy  where  the  ar©  arranged  In 
order  of  sis©*  so  that  x]L  < xp  < < x^  < x^©  Now  a logical 


} 

' 

, 


, 


measure  of  the  distane©  between  th©  largest  v&lu©  and  th© 
other  four  values  Is 


. *5’*k 
1 k5"3!i 


Cl) 


io.^oe  th©  proportion  of  th©  total  range,  that  the  distane© 
between  th©  suspected  value  and  its  adjacent  value*  is0 

Now  Perry  records  with  what  frequency*,  among  his  millions 
of  sets  of  five  values  each*,  different  values  of  r„  ©ecuPo 
He  finds  that  a value  of  r^  larger  than  yjBQ  occurs  on©  time  in  one 
hundred o H©  then  reasons  this  ways 

n!  have  found  that  among  sets  of  five  observations  each 
(containing  no  mistakes)  a value  of  r^  larger  than  0?8 0 is 
quit©  unlikely  (occurs  only  one©  in  on©  hundred) 0 If  nowg  in 
ay  future  experiments  I get  a set  of  five  observations  for 
which  Pj  is  larger  than  o?80  I will  conclude  that  my  largest 
observation  is  in  erroro^ 

!°3  CONFIDENCE  IN  THE  'lESTs  This  seems  reasonable 0 But 
what  confidence  can  Perry  have  in  such  a procedure?  How  often 
will  h©  consider  as  mistaken  a perfectly  good  observation? 

How  often  will  he  consider  acceptable  an  incorrect  observation? 

dearly  from  th©  way  h©  derived  th©  test9  he  will  classify 
a perfectly  good  largest  observation  as  mistaken  one©  among 
©n©  hundred  sets  of  fiv©  each*,  ©n  the  average o But  there  is 
n©  general  answer  to  th©  question  of  how  often  he  will  let 


pass  a mistaken  observation o This  depends  on  how  33mi  stakes1* 
the  mistaken  observation  is®  If  a very  large  error  w©r©  made^ 
his  test would  tend  fc©  reject  the  observation  almost  certainly© 
If  a very  small  error  were  made^  his  test  would  tend  to  reject 
the  observation  with  a small  probability© 

The  following  chart  [1]  gives  some  idea  of  the  perform 
mane®  of  in  detecting  mistaken  observations©  It  is  based 
on  a sampling  experiment  in  which  samples  of  five  from, 
a normal  population  with  mean^u  standard  deviation  o wei’e''  v 

contaminated  with  values  drawn  from  a normal  population  with 
mssn  yu  *>£  o and  standard  deviation  aQ  The  ordinate  a hows  the 
percent  discovery  of- cpntaminatora  {the  proportion  of  tho  time 
the  c on tami mating  population  provides  an  extreme  value  and 
the  tost  discovers  this  value)  while  the  abscissa  shows  X s the 
magnitude  of  the  shift  (error)  of  the  contaminator  in  stands 
ard  deviations© 


o 1 1 3 !i  M ? 3 ^ 


(Ft©m  Dixon0®  article  [XJ) 
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W©  said  above  that  once  in  every  100  sets  of  Tallies 
(on  the  average ) Perry  would  consider  as  mistaken  a perfectly 
good  observation©  If  h©  were  to  reject  this  observation© 
and  then  compute  the  mean  and  standard  deviation  of  the  remain- 
ing values,  these  would  be  biased  estimates©  Also  when  a good 
observation  is  rejected©  any  further  statistical  tests  of 
significance  will  become  less  reliable®  Shis  la  the  price 
that  he  must  pay  for  improving  the  data  in  the  cases  where 
a mistaken  observation  is  removed 0 

l ok  HA  THEM  (El  CAL  DERI  VA  TI  ON  % Of  course©  o780©  the  value 
of  r..:,  exceeded  by  chance  1 percent  of  the  time  f- called  the  1 
percent  level  of  significance  of  x»,  )8  is  not  determined  by 

actually  making  millions  of  sets  of  five  observations  eacho 

•+  . * 

Bather  it  i my  be  calculated  mathematically  [2j  with  greater 

accuracy  than  if  millions  of  sets  of  five  observations  had 
been  used©  Ih©  basis  assumption  is  that  repeated  measurement® 
would  follow  the  normal  dis trlbution0 

Smallest Observation  Suspected®  What  if  Perry  suspects 
the  smallest  observation  in  a set  (of  five©  say)? 

In  this  case©  he  computes 

x <=>  x 

r « 2 1 

8 x5  * xi  (2) 

H©  comp  axes  i t with  0jQQe  as  before®  Exactly  th©  asm©  reason® 
ing  applie  s a 
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lo5>  Table  X©  Let  us  now  define  r as  either  r*.  or  r 

, Is 

depending  on  whether  the  largest  or  smallest  of  a set  is  being 
tested©  Table  1 at  the  ©nd  of  the  paper  gives  R*  the  value 
of  r at  various  significance  levels*  ag  for  sample  sis©  n from 
3 t©  30© 

Thus  for  example®  for  n s 8 and  a * ©05*  the  table  gives 
R ® ©4^9 © This  moans  that  in  100  sots  of  8 observations  each 
fro©  of  mistakes*  five  values  of  r^  will  b©  larger  than  ©468 
and  5 values  of  r will  b®  larger  than  ©468  (on  the  average)© 

Why  &r©  various  significance  levels  given?  The  reason 
is  that  no  one  significance  level  is  appropriate  to  all  prob- 
lems© For  example  consider  these  two  cases? 

fa)  Additional  observations  are  not  possible© 

(b)  Additional  observations  are  possible© 

In  case  (a)  for  many  problems  it  might  be  appropriate 
to  compute  r and  test  it  at  the  1 percent  level  of  signifies  me®© 
If  the  observed  value  ©f  r is  larger  than  the  tabl©  value  for 
a 53  1 percent^  It  might  then  b©  a good  idea  to  e xclude  the 
responsible  observation© 

In  ©as©  (b)®  for  many  situations a reasonable  procedure 
might  b@  to  test  r at  the  5 percent  level©  If  the  sample 
value  of  r Is  significant'  at  the  5 percent  level*  on©  or  more 
additional  observations  would  be  taken©  If  the  observation 
originally  suspected  remained  outlying*  it  would  b©  tested 
again*  using  the  combined  set  of  observations©  This  time* 
however*  the  r test  would  b©  performed  at  the  1 percent  level 


6 


of  significance®  If  the  outlier  were  significantly  deviant 
at  the  X percent  level 0 It  would  be  re  jee t@d0  It  should  b© 
noted  that  among  many  sets  tested  this  way0  the  proportion 
of  perfectly  good  largest  values  rejected  this  way  will  b® 
less  than  1 pereento  This  is  because  the  observation  has  a 
^second  chance'®8  before  it  is  finally  rejected© 

le.6  SUMMARY?  A set  of  n obsirvations  is  made0  Ho  pro® 
vioua  data  are  available  from  which  to  estimate  the  variability 
of  a measurement 0 What  is  a rational  procedure  for  tasting 
whether  the  largest  Cor  smallest)  of  the  set  is  too  deviant 
to  b©  explained  by  the  ordinary  errors  of  measurement? 

Consider  the  n observations  in  order  of  sis©* 


*!  i x2  i *•*  i xn 

Compute 

X - X _ 

r ® ,,n  aT1  (if  x suspected) 

3n  - X1 

©r 

r ® ^ «i  (if  tj  is  suspected) 

*n  - X1 

Tab!©  I may  use  t©  determine  how  likely  it  ia  to  get 
as  large  a'  va’ue  of  r as  actually  obtained^  simply  by  chance® 
A procedure  that  might  he  appropriate  for  many  problems  in 
as  follow? 

fa)  Ho  additional  observations  possibles  In  this  case., 
cozsparr  the  computed  r with  tho  Table  I value  at  the  1 p©r~ 
cent  -oyelo  If  the  computed  value  of  r is  larger  than  the 
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Table  1 value*  exclude  the  deviant  observation©  Otherwise* 
do  noto 

(b)  Additional  observation®  possible s In  this  case® 

compare  the  computed  r with  the  table  value  of  r at  the 

5 percent  level©  If  th©  computed  value  of  r is  larger  than 

* 

th©  table  value * tak©  on©  or  more  additional  observations 
(depending  on  convenience 9 cost  of  observations*  etc0)o 
Otherwise  accept  th©  suspected  value  without  taking  additional 
observations 9 

In  in  th©  enlarged  set  (containing  all  the  original  and 
'the  additional  observation (s) ) 0 th©  previously  suspected  value 
remains  outlying*  compute  r for  the  enlarged  set o This  time 
compare  It  with  the  Table  I-  va3.ua  at  th©  1 percent  level© 

If  th©  computed  value  exceeds  th©  table  value*  exclude  th© 
outlier?  otherwise  do  no to 

L?  EXAMPLES  i 1®  Anna  Lis  to*  chemists  determines  five 
values  for  the  iron  content  ©f  an  unknown  solution  by  a new 
process:  ? olj29  7 olid ? o 39 0 7a6l0  7©  44  (in  percent)©  She 
suspect®  ?o61  as  being  mistaken*  since  it  is  so  much  larger 
than  th©  other  value s«  However  she  cannot  determine  additional 
values  because  no  more  samples  are  available©  How  can  she 
decide  whether  to  exclude  7e>oX  or  notf 

Sine©  no  previous  data  ar©  available  from  which  to  compute 
th©  precision  of  measurement*  th©  r test  is  appropriate©  The 
first  step  is  to  arrange  th©  five  values  in  order  of  sis.es 
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7 <,39*  7-42*  7.44.  7-48*  7-61 

I 

Then 


r es 


x5  " x4  . 
*5  * X1 


591 


Sins©  this  Is  lees  than  ©?809  the  ©01  point  of  r fox*  n ® 5® 

Anna  List  retains  th©  suspected  valu@0  7 ©6i o 

2©  H©  Go  Neer  fires  projectiles  froya  a gun  under  constant 

s©nd Iti ©ns o Here  are  the  distances  in  feet  (in  order  of 

6801  7683 

. Til 2k  7720 

7502  7799 

7$kh 


He  suspects  6801  as  being  inconsistent  with  the  other  values  © 
What  shall  h©  do? 

He  compute® 


7424  - 6801 


S a&2hr 


This  value lies  between “the  ©01  and  th©  ©05  points  of  r for 
n 53  ? © Hence  Ie  G©  leer  fires  an  additional  round  and  gets 
a new  value  of  7603© 

Sine©  6801  remains  outlying  in  th©  enlarged  set  ©f  ©lghte 
he  computes  r for  this  set  of  eight-,  Obviously  r remain® 

0624®  "This  time  it  is  larger  than  the  ©01  point  ©f  r for  n g 8a 
Hence  No  Go  Keer  rejects  6801  and  uses  only  th®  remaining  seven 
values© 
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iL Estimate  of  Measurement  Variability  Available © 

In  a great  many  laboratory  situations^  past  data  are  available 
for  estimating  the  under  tain  ty  of  a measurement^  Tims  if  a 
laboratory  makes  routine  analysis  in  triplicate  of  incoming 
Bpeeimsntsd  there  will  b©  sets  of  triplicate  values  from 
which  to  estimate  th®  precision  ©f  measurement©  It  Is  clear 
that  'inhere  such  information  is  available 8 it  should  b©  used, 
in  deciding  whether  an  outlier  is  mistaken  or  not,  This  will 
make  th®  decision  more  reliable  than  if  only  th©  on©  set  con- 
taining the  suspected  value  is  used© 

2 el  u Taste  The  test  ratio  used  now  la 


U « 


n 


(it  x is  th© 


suspected  value) 

(k) 


or 


where 


x * zi 


(if  is  th®  suspected  value)  © 


s ® standard  deviation  of  an  individual  measurement, 
^ based  on  d degrees  of  fVoedom© 


2©2  Calculating  s^o  To  determine  s from  a single  set 
of  measurements  we  would  first  calculate  th©  sum  of  th©  squares 
of  the  deviations  of  th©  observations  from  their  mean©  Then 
tre  would  divide  by  one  less  than  the  number  of  observations© 


This  would  give  us  8,%  Thus 

d a ■ 

p 2(*i=S3 

a,"*  1=1 
d • — 3^— 


(5) 
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On  the  other  hands  suppose  a number  of  sets  of  observations 
were  available s 


at 

Now  we  could  calculate  from 


SETS 

OBSERVATIONS 

MEAN 

^OStseSSSSSSj 

1 

- . 

Xxi*' 

^12* 

°°°e  Xln% 

X1 

2 

X21* 

*22* 

x&s,z 

tm 

Z2 

k ■ 

• 

XkV 

Xk2° 

0 0 ol  X,  „ 

is., 

k 

*.  *—*  (6) 
'~**OTTS^i)  4.  (ng«l)  ❖coo*?’  (njr^l)  . _ 


&0  the  member  of  degrees  of  freedom  for  estimating  th©  uncertain*-- 
ity  of  measurements  is  (n^«l)  ❖ (n^l)  ❖ ❖ (n^l)  0 

2q3  EXAMPLES  An 'example  will  make  th©  whole  procedure 


clean 

D r?i  Kern  Mist  has  just  run  a set  of  three  determinations 
on  the  yield  of  an  ore*»refining  process & Hie  values  are? 

39o35*  39o30#  39oOOs  with  mean8  3c  « 39o22o 
H©  wished  to  test  whether  the  M39oOOH  is  unusually  deviant'o  * He 
has  past  data  to  estimate  the  precision  of  measurements 
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Analysis 


Set 

1 

MB* 

2 

■ 1 

. 1. 

36o5i 

' 36 .57 

36=70 

2 

30®  27 

30=35 

• ■ 30ol9 

3 

35o©0 

. 35=53 

‘ 35=36  ■ 

4 

'43.51' 

43.65 

43.65 

5 

5io06 

Slol? 

51=00 

6 

48«03 

38=19 

iffi<,31 

T 

39®  ^7 

39=51 

39.36 

8 

33.46 

33=21 

33=28 

Although  he 

could 

calculate 

2 ' 

from  (6) 

convenient  (especially  with'  a computing  machine' callable) 

t©  tisa  (7) : 
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The  u^ratio  defined  fey  (2jJ  gives? 


u _ 


X ■* 


Si  as  39 o22  - 39,00  , 1„S8 


®d  ol39 

H©  now  uses  Table  It  which  give  a the  $ percent  and  1 p©r<=> 

eenfc  levels  of  u for  various  values  of  n and  do  q Is  the 

sis©  of  th©  sample  which  contains  the  suspected  value ^ While 

d is  the  number  of  degrees  of  freedom  on  which  s is  based e 

a 

In  th©  present  casi>  n a 3 ©nd  d “ I60 

The  observed,  value  of  u®  X© $8»  is  considerably  loss  then 
the  table  value  of  u at  th©  5 percent  lev©!®  Xo9©o  Henc©  h© 
concludes  that  the  suspected  value  39®  GO  is  not  significantly 
outlying©  In  other  words  the  deviation  of  39® 00  from  the  moan 
©f  the  set  of  thro®  -measurements  is  easily  explainable  in  terms 
of  fch©  precision  of  the  measurement  process©  Henc©9  39oQ0 
is  accepted  into  the  fold  of  good  measurements© 

2 oh  TABLE  Zo  When  past  data  are  available  * the  u^r&fci® 
may  b@  computed  and  Table  2 used  just  th©  r^ratio  and  Table 
I were  used  for  th©  case  where  no  past  data  were  available© 

Th©  procedure  outlined  in  paragraph  1©5  for  fch©  two  cases  (a) 
and  (b)  may  be  .followed  just  as  before  (using  u and  Table  2 
instead  of  r and  Table  I)« 

III©  Cautions  and.  Comments  © a©  Obviously®  if  the  ©xperi® 
metor  knows  that  a mistake  has  occurred  he  should  reject  th® 
observation©  Th©  tests  of  this,  paper  ar©  used  only  if  h® 
doesn8t  know  that  a mistake  has  occurred© 
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bo  If  the  experimenter-  uses  this  technique  for  a certain 
routing  type  of  measurement®  h©  should  apply  It®  implicitly 
or  expliclty®  every  time  he  makes  that  type  of  measuremento 
After  several  explicit  applications  of  this  technique®  h© 
will  probably  b©  able  to  perform  the  r(or  u)  test  in  all  but 
th®  most  doubtful  cases  without  actually  explicitly  doing  the 
arithmetic®  since  he  will  have  the  critical  value  of  r(or  u) 
in  mlndo  H©  should  not®  however®  reject  outlier-®  by  th©  v 
test  in  some  cases®  and  accept  others  just  as  badly  deviating^ 
simply  because  h©  did  not  apply  th©  test  in  these  latter 
q as© So 

Co  Both  the  r and  u tests  are  based  on  th©  assumption 
that  repeated  measurements  of  th©  same  unknown  follow -th© 
normal  frequency  dlstrlbutlono  If®  in  actual  practice®  the 
distribution  of  repeated  measurements  is  markedly  different 
from  the  normal  curve®  than  the  us©  of  these  tests  will  lead 
to  different  risks  than  originally  Intended© 

d0  The  use  of  th©  0Q1  and  points  is  arbitrary,©  Th© 
individual  experimenter  should  use  whatever  levels  of.  signif- 
icance are  most  appropriate o 

Qc,  Suppose  th©  type  of  measurement  is  such  that  th® 
suspected  value  is  practically  always  the  smallest  in  th© 
set  (for  example® . a chemical  analysis  where  some  of  tha 
material  may  b©  washed  out)®  or  practically  always  th®  large  a to 
Then  as  stated  above®  1 percent  of  th©  time  a perfectly  good 
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observation  will  b©  rejected  in  case  (a)  of  paragraph  lc*>Q 
Suppose  however  the  type  of  measurement  is  such  that  th©  sus~ 
pae ted  value  may  be  either  the  largest  or  the  smallest*,  In 
this  ease  about  2 percent  of  the  time  a perfectly  good  observa® 
tion  will  be  rejected*  The  appropriate  tabular  point  should 
b©  selected  with  this  in  mind* 

fo  Other  tests  for  rejection  of  suspected  values  are 
available  [ljc  However ^ the  r and  u tests  have  been  selected 
because  of  their  ease  of  application., 

go  The  whole  question  of  rejection  of  suspected  values 
is  a difficult  one*,  and  has  given  rise  to  quit©  a bit  of 
discussion*  [See  references* 3 If  the  scientist  or  engineer 
at  the  National  Bureau  of  Standards  is  In  doubt  about  any 
assumption*  procedure * or  conclusion  involved  in  a specific 
practical  problem*  we  suggest  that  he  consult  the  Statistical 
Engineering  Laboratory  (11*33 o 
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Upper  per  cent  points  of  the  studentised  extreme  deviate 
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I088  2ol2  So 28  2.41  2,51  2o59  2,66  2o50  2C?7  2,95  3,08  3,18  3,2?  3,34 

1-87  2oXl  2o2?  2o39  2^49  2,58  2,65  2.49  2*75  2,92  3«06  3.16  3o2ii  3*31 

log?  2ol0  2o26  2o38  2J18  2, ,56  2,63  2*47  2o?3  2,91  3o04  3dk  3C2 2 3,29 

le>o4  2o0?  2c23  2o35  2044  2«52  2,59  2o43  2,68  2,85  2C9?  3®©7  3d5  3<,2 2 

I082  ^o04  2c 20  2,31  2c40  2o48  2,55  2o38  2e62  20?9  2,91  3,01  3*08  3d5 

I08O  2,02  2ol7  2*28  2,37  2c-44  2,51  2*3^2*57  2.73  2.85  2,94  3o02  3o08 

fo?8  lo99  2.14  2.25  2o33  2o4l  2,4?  2,30  2o92  2,68  2,79  2088  2,95  3C01 

}°27  ?oll  2o2i  2o3®  2,43  2,25  2,43  2 062  2o73  2 082  B089  2o95 

lo?4  1.94  2o08  2 ©l8  2o2?  2033  2.39  2,22  2,43  2.5?  2,68  2*76  2,83  2088 
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THE  NATIONAL  BUREAU  OF  STANDARDS 


Functions  and  Activities 

The  National  Bureau  of  Standards  is  the  principal  agency  of 
the  Federal  Government  for  fundamental  and  applied  research  in  phys- 
ics,  mathematics,  chemistry,  and  engineering.  Its  activities  rang© 
from  the  determination  of  physical  constants  and  properties  of  ma- 
terials, the  development  and  maintenance  of  the  national  standards 
of  measurement  in  the  physical  sciences,  and  the  development  of 
methods  and  instruments  of  measurement,  to  the  development  of  special 
devices  for  the  military  and  civilian  agencies  of  the  Government. 
The  work  includes  basic  and  applied  research,  development,  engi- 
neering, instrumentation,  testing,  evaluation,  calibration  services, 
and  various  scientific  and  technical  advisory  services..  A major 
portion  of  the  NBS  work  is  performed  for  other  government  agencies, 
particularly  the  Department  of  Defense  and  the  Atomic  Energy  Com- 
mission. The  functions  of  the  National  Bureau  of  Standards  are  set 
forth  in  the  Act  of  Congress,  March  3,  1901,  as  amended  by  Congress 
in  Public  Law  619,  1950.  The  scope  of  activities  is  suggested  in 
the  listing  of  divisions  and  sections  on  the  inside  of  the  front 
cover. 

Reports  and  Publications 

The  results  of  the  Bureau’s  work  take  the  form  of  either  actual 
equipment  and  devices  or  published  papers  and  reports.  Reports  are 
issued  to  the  sponsoring  agency  of  a particular  project  or  program. 
Published  papers  appear  either  in  the  Bureau’s  own  series  of  publi- 
cations or  in  the  journals  of  professional  and  scientific  societies. 
The  Bureau  itself  publishes  three  monthly  periodicals,  available 
from  the  Government  Printing  Of fice : the  Journal  of  Research,  which 
presents  complete  papers  reporting  technical  investigations;  the 
Technical  News  Bulletin,  which  presents  summary  and  preliminary  re- 
ports on  work Tn  progress;  and  Basic  Radio  Propagation  Predictions, 
which  provides  data  for  determining  the  best  frequencies  to  use  for 
radio  communications  throughout  the  world.  There  are  also  five 
series  of  monperiodieal  publications:  the  Applied  Mathematics  Se- 
ries, Circulars,  Handbooks,  Building  Materials  and  Structures  Re- 
ports, and  Miscellaneous  Publications. 

Information  on  the  Bureau’s  publications  can  be  found  in 
NBS  Circular  460,  Publications  of  the  National  Bureau  of  Standards 
($1.00).  Information  on  calibration  services  and  fees  can  be  found 
in  NBS  Circular  483,  Testing  by  the  National  Bureau  of  Standards 
(25  cents).  Both  are  available  from  the  Government  Printing  Office . 
Inquiries  regarding  the  Bureau’s  reports  and  publications  should  be 
addressed  to  the  Office  of  Scientific  Publications,  National  Bureau 
of  Standards,  Washington  25,  D.  C. 
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